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(54) Defect correction in electronic imaging system 



(57) A method of processing a video data stream 
comprising a series of pixel values corresponding to pix- 
el sites in an electronic imaging device so as to correct 
defective pixel values comprises filtering the video data 
stream in real time so as to correct or modify defective 
pixel values, based on the values of a plurality of neigh- 
bouring pixel values. The filtering of each pixel value us- 
es the value of the current pixel as part of a dataset in- 
cluding the values of neighbouring pixels in determining 
whether and/or how to correct or modify the current pixel 



value. Those pixel values which are most severely de- 
fective are identified and stored. A first filtering algorithm 
is applied to those pixels whose locations are not stored 
and a second f i Itering algorithm is applied to those pixels 
whose locations have been stored. A preferred filtering 
algorithm comprises sorting the values of the current 
pixel and of the neighbouring pixels into rank order and 
modifying the current pixel value on the basis of its place 
in said rank order. Apparatus for implementing the meth- 
od is also disclosed. 
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Description 

[0001] The present invention relates to methods and 
apparatus tor correcting detects in video data generated 
by electronic imaging systems. The invention is most 
particularly concerned with the correction of detects 
arising from defective pixel sites in electronic image sen- 
sors, but is also applicable to more general noise reduc- 
tion in video data streams. The invention is equally ap- 
plicable to monochrome and colour video data and may 
be useful in relation to still imaging systems as well as 
kinematic video systems. 

[0002] The majority of electronic imaging devices are 
now implemented using semiconductor technologies. 
Examples include the CCD, which is implemented using 
a form of MOS manufacturing process, and, more re- 
cently, image sensors manufactured using standard 
CMOS semiconductor processes. 
[0003] In all of these cases, the sensor normally com- 
prises a 1- or 2-dimensiona! array of discrete pixels. 
[0004] It is in the nature of the manufacturing proc- 
esses employed in the production of such devices that 
occasional defects occur at individual pixel sites. Such 
defects may variously cause the affected pixel to be un- 
rcprcscntativcly brighter or darker than the true image 
at that point (including the extreme cases of saturated 
white or black pixels). 

[0005] Such defects affect some proportion of the 
population of individual imaging devices ("chips") on 
each manufactured wafer. Chips so affected must nor- 
mally be rejected for use unless the defects can in some 
way be masked or corrected. It is economically attrac- 
tive to mask or correct defective pixels enabling other- 
wise rejected chips to be passed as good product. This 
improves the apparent yield of good imaging chips per 
wafer and thereby lowers the cost per usable chip. 
[0006] It is known in the art to calibrate imaging de- 
vices at the point of camera manufacture : so that the 
locations of defective pixels in the imaging array are 
identified and stored. In subsequent use of the device, 
pixel data from these locations is in some way masked 
or corrected in the live video data stream. 
[0007] One simple and well known maskingtechnique 
is to substitute for the defective datum a copy of the val- 
ue of a neighbouring pixel. More sophisticated tech- 
niques are also possible and typically may produce an 
estimate of the correct value of the defective pixel data 
by applying an algorithm to the data obtained from the 
neighbouring pixels in one or two dimensions. General- 
ly, the best correction filters use a mixture of linear and 
non-linear estimators and work on at least a 3 x 3 pixel 
neighbourhood centred on the defective pixel. 
[0008] This prior technique of calibrating individual 
sensors at the point of manufacture has two main dis- 
advantages. Firstly, and most significantly, the process 
of calibrating the sensor to determine defect locations 
is an inconvenient and expensive manufacturing bur- 
den. Secondly, defects may sometimes be transient in 



nature, so that defects present and corrected for at the 
time of calibration may subsequently disappear- or, 
worse, new defects may occur subsequent to calibra- 
tion. These latter defects will remain uncorrected in sub- 

5 sequent camera use and will show clearly as blemishes 
on the images output by the camera. . 
[0009] It is a first object of the present invention to pro- 
vide methods and apparatus for the correction of defects 
in electronic imaging systems which obviate or mitigate 

io the above mentioned disadvantages of prior art image 
defect correction schemes. 

[0010] Whilst the invention may be implemented us- 
ing known error correction algorithms for correcting the 
pixel values output by defective pixel sites, it is a further 
15 object of the invention to provide improved methods and 
apparatus for filtering video data signals, both for the 
purpose ol correcting image defects originating from de- 
fective pixel sites and for more general noise reduction 
purposes. 

20 [001 1 ] The invention, in its various aspects, is defined 
in the Claims appended hereto. Other features and as- 
pects of the invention and of the preferred embodiments 
thereof will be apparent from the following description. 
[0012] Embodiments of the invention will now be de- 

25 scribed, by way of example only, with reference to the 
accompanying drawings, in which: 

Fig. 1 is a block diagram illustrating a first embodi- 
ment of the invention; 

30 

Fig. 2 is a block diagram illustrating a preferred em- 
bodiment of the invention; 

Figs. 3(a) and 3(b) are illustrations representing pix- 
35 el neighbourhood locations employed in correcting 
image defects; 

Fig. 4 is a more detailed block diagram illustrating 
a particularly preferred implementation of the em- 
40 bodiment of Fig. 3; and 

Fig. 5 is a graph illustrating the operation of a digital 
filter employed in the embodiment of Fig. 4. 

45 [0013] Referring now to the drawings, Fig. 1 illustrates 
a first, most general embodiment of the invention. 
[0014] An image sensor 10 of known type comprises 
an array of pixels. The sensor array 10 outputs an ana- 
logue data stream which is converted to digital form by 

50 analogue to digital conversion means 1 2. Assuming a 
two dimensional pixel array, the data stream comprises 
a series of pixel values output line by line from the sen- 
sor 10. The digital data stream would normally be en- 
coded by encoding means 14 in a manner to suit the 

55 intended end use of the video data. 

[001 S] In accordance with the present invention, the 
live video data stream is filtered in real time by digital 
filter means 1 6 so as tocorrect or mask anomalous pixel 
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values which are judged to arise from defective pixel 
sites in the sensor 10. Typically, the filter 16 judges a 
pixel value to be defective if it is significantly higher or 
lower than its neighbours in either one or two dimen- 
sions. The filter replaces the defective pixel value with 
a substitute value. The substitute value may be derived 
by any suitable algorithm, which may involve linear and/ 
or non-linear processes which may operate on sur- 
rounding pixel data from a one- or two-dimensional 
neighbourhood surrounding the defective pixel value. 
[0016] The filter 16 works permanently on the normal 
sensor output and does not require the use of any ref- 
erence scene or predetermined calibration data. Rather, 
the filter depends on predetermined criteria for identify- 
ing defective pixel values in the live data stream and on 
predetermined rules for deriving substitute pixel values 
to replace the defective pixel values. 
[0017] This "live" or "in-line" correction of defective 
pixels overcomes the manufacturing burden of prior art 
techniques and deals automatically with defects which 
arise after manufacture. It further provides a degree of 
noise filtering on noisy images : correcting excessively 
large single-pixel noise spikes. 

[0018] Applying automatic correction in this way to an 
entire image can s in some circumstances, cause an-un- 
desirable deterioration in the overall image quality un- 
less the correction filter is constrained in severity. This 
limits the effectiveness of the technique in its most basic 
form. 

[0019] The Applicant has found that the most suitable 
class of pixel-correcting filter is one which uses the cen- 
tral pixel data itself as part of the data set used to deter- 
mine the correction to be applied. Typically, this means 
that the non-defective portions of the image (that is, the 
majority of each image) are unaffected by the presence 
of the correcting filter. The filter will, however, correct 
defects of large magnitude. 

[0020] Unfortunately many defects which it would be 
desirable to correct are not of large magnitude. Typical 
examples are pixels with a significant gain error, or pix- 
els which are stuck at an intermediate image value. It 
has not been found to be possible to devise a single filter 
which is capable of correcting for these more subtle de- 
fects which does not also falsely correct many non-de- 
fective pixels in a manner which has an undesirable ef- 
fect on the overall image, such as by producing a smear- 
ing effect. 

[0021] Fig. 2 illustrates a preferred embodimenl of the 
invention, in which the single filter 16 of Fig. 1 is replaced 
by first and second filter stages 18 and 22 and a defect 
memory or database 20. In accordance with this 
scheme, the first filter stage 18 performs two functions. 
Firstly it applies a more subtle correction algorithm to 
the complete data stream, so as to correct defects of 
lower magnitude as noted above. Secondly, it identifies 
pixels exhibiting more extreme defects, and passes in- 
formation regarding these pixels to the defect memory 
20, which stores information regarding those pixels 



which are judged to be most severely defective. The de- 
fect memory 20 controls the operation of the second fil- 
ter stage 22, which applies more severe correction se- 
lectively to those pixels identified in the defect memory 

s 20. Typically the number of pixels for which severe cor- 
rection is required will be less than 1 % of the total pixel 
count. The pixel locations stored in the defect memory 
20 are restricted to those that, historically, appear to be 
most severely in error as detected by the first filter stage 

to 18. 

[0022] That is, for each video frame (or for each still 
image captured by the sensor), all defectsare monitored 
by the first filter stage 18 and those pixel locations ex- 
hibiting the largest apparent errors are added to the de- 
15 feet memory 20, if not already identified and stored. 
[0023] In order to enable the contents of the defect 
memory 20 to remain dynamic over time, a manage- 
ment strategy is required so that locations representing 
transient noise defects or defects which disappear over 
20 time can be identified and removed from the defect 
memory 20. Besides preventing future correction of 
non-defective pixel values, this also creates memory 
space for new or previously undetected defects (the 
memory space 20 is necessarily limited and it is desir- 
es able that it be as small as possible consistent with the 
number of defects which are likely to be encountered in 
practice). 

[0024] Typically, the defect memory 20 might store 
less than 1% of ail possible pixel locations. Accordingly, 

30 no more than 1% of pixels will be subject to severe cor- 
rection. This proportion is so low as to be unnoticeable 
to a human observer of the corrected video or still image. 
[0025] A preferred embodiment of the scheme illus- 
trated in Fig. 2 will now be described with reference to 

35 Figs. 4 and 5. 

[0026] Referring firstly to Figs. 3(a) and 3(b), these 
illustrate examples of "pixel neighbourhoods" operated 
on by digital filters of the type employed in the invention. 
In a two-dimensional pixel array, each pixel (neglecting 

40 the pixels at the edges of the array) is surrounded by 
eight immediately neighbouring pixels, forming a 3 x 3 
array. The particular pixel operated on by a filter at any 
point in time is the central pixel p(c) of the 3 x 3 array. 
Fig. 3(a) illustrates the situation when the filter includes 

45 the central pixel value along with the values of the sur- 
rounding eight pixels in the dataset employed to deter- 
mine a substitute value for p(c). Fig. 3(b) illustrates the 
situation when the filter excludes the centra! pixel value 
from the dataset employed to determine a substitute val- 

50 ue for p(c). These two alternatives are both employed 
in the two stage filtering provided by the preferred em- 
bodiments of the present invention, as shall be de- 
scribed in greater detail below. It will be understood that 
the use of a 3 X 3 array for the filter dataset is merely 

55 an example., being particularly applicable to mono- 
chrome image sensors. Larger and/or differently orient- 
ed arrays may be appropriate in some circumstances, 
particularly for colour sensors, and the approach de- 
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scribed in the present example can clearly be extended 
to other shapes or sizes ot array. 
[0027] Referring now to Fig. 4, there is shown a block 
diagram ot a video data filtering system corresponding 
to blocks 18, 20 and 22 of Fig. 2. The input data stream 
consists of a series of input pixel values p(in) and the 
output datastream consists of a series of output pixel 
values p(out). 

[0028] The input datastream is firstly sampled by a 
sampling network consisting of line memory buffers 30 
and 32, each of which is capable of storing a complete 
line of video data, and individual pixel value memory 
buffers 34, 36, 38, 40, 42 and 44. The incoming video 
signal is routed through the line buffers 30, 32 and into 
the pixel buffers 34 - 44 so that, over a number of clock 
cycles, nine pixel values tor the central pixel p(c) and 
surrounding neighbours are accumulated to be operat- 
ed on by the filter system. The line buffers 30, 32 suitably, 
comprise random access memory, while the pixel buff- 
ers 34 - 44 may be D-type flip-flops. 
[0029] The central pixel value p(c) is extracted on line 
46 as shown, while the eight neighbouring values are 
applied to block 48. Block 48 sorts the values of the 
neighbouring pixels into rank order according to their 
amplitudes, and outputs the values in rank order, with 
the highest value output on the upper output line 48U 
and the lowest value on the lower output line 48L. In this 
example, the fitter system only employs the h ighest, low- 
est and middle two ranking values out of the eight input 
values. However, variations on this example could uti- 
lise other combinations of the eight ranked values, as 
shall be discussed further below. 
[0030] The ranked values of the neighbouring pixels 
are employed by both the first and second stage filter 
processes 1 8 and 22 of Fig. 2. In fact, the two filter stag- 
es share components and functions of the embodiment 
illustrated in Fig. 4, rather than being discrete systems 
as shown in Fig. 2. However, their essential functionality 
is separate and is in accordance with the schematic rep- 
resentation provided by Fig. 2. 

[0031] The first stage filtering operates to apply rela- 
tively subtle correction to the entire data stream while at 
the same time identifying defect locations to which the 
second stage filtering is to be applied, as follows. 
[0032] The highest and lowest ranked pixel values on 
lines 48U and 46L and the central pixel value p(c) on 
line 46 are input to block 50, which operates as a "one 
from three" multiplexer. Block 50 compares p(c) with the 
highest and lowest ranked values. If the value of p(c) is 
greater than the highest ranked value then the highest 
ranked value is output from block 50, replacing p(c) in 
the data stream. If the value of p{c) is less than the low- 
est ranked value then the lowest ranked value is output 
from block 50, replacing p(c) in the data stream. If the 
value of p(c) is less than the highest ranked value and 
greater than the lowest ranked value, or is equal to either 
value, then the value of p(c) is output from block 50, so 
that p(c) is unaffected by the first stage filter. 



[0033] This filtering scheme is illustrated in Fig. 5, in 
which the rank of the input pixel value is plotted against 
the rank of the pixel value which is output by the filter. 
The nine ranks of this example are numbered from -4 to 

5 +4, with zero being the rank of the median pixel value. 
The graph shown corresponds to the scheme described 
above. If p(c) is ranked +4 then it is replaced by the value 
of rank +3. If p(c) is ranked -4 it is replaced by the value 
of rank -3. Otherwise it is unaffected by the filter. 

io [0034] The filter could be modified to allow maximum 
values restricted to ranks 1 or 2, as indicated by the dot- 
and-dash lines, in which case different outputs from 
block 48 would be employed. The filter could also be 
made to be switchable between these different modes 

is of operation if required. The horizontal axis of Fig. 5 cor- 
responds to a "median filter", in which the median value 
is output regardless of the input value. The diagonal line 
through the origin indicated by the dashed lines corre- 
sponds to zero filtering, in which the output is always 

20 equal to the input. 

[0035] Since this filtering operation is applied to the 
entire data stream, it acts as a general noise reduction 
filter as well as correcting relatively subtle defects aris- 
ing from defective pixel sites in the sensor array. As such 

25 \\ is potentially useful in applications other than that il- 
lustrated in Figs. 2 and 4. For example, it could be em- 
ployed purely as a noise reduction filter in imaging sys- 
tems using prior art calibration schemes to correct sen- 
sor defects. This filtering scheme will be referred to 

30 hereinafter as a "scythe filter" and its output value as the 
"scythe value". 

[0036] The second stage filtering 22 of Fig. 2, in this 
example, is based on the median value of the pixels 
neighbouring the central pixel p(c). A conventional me- 

35 dian filter applied to a 3 x 3 array would output a value 
corresponding to the median value of the nine pixels in 
the array. In the present case, it is preferred to neglect 
the value of the central pixel, since this has already been 
presumed to be erroneous when the second stage fil- 

40 tering is applied. Accordingly, a median value is calcu- 
lated based on the values of the eight neighbouring pix- 
els, excluding the central pixel p(c) as shown in Fig. 3 
(b). Since there is an even number of neighbouring pix- 
els, the median value used is the mean value of the two 

45 middle ranking pixel values. The sorting of the neigh- 
bouring pixel values into rank order, described above, 
facilitates this. As seen in Fig. 5, the values of the two 
middle ranking values output f rom block 43 are summed 
and divided by two, to provide a pseudo-median value. 

so [0037] This filtering scheme will be referred to herein- 
after as a "ring median filter 1 ' and its output as the "me- 
dian value". 

[0038] In the example of Fig. 4, it can be seen that 
scythe (first stage) filtering and ring median {second 
55 stage filtering) both take place in parallel on the entire 
data stream. Both the scythe and median values are in- 
put to a final "one from two" multiplexer 52, the final out- 
put p(out) being determined by the contents of the defect 
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memory 20 of Fig. 2. if the pixel location corresponding 
to the central pixel p(c) is stored in the defect memory 
20, then multiplexer 52 will select the ring median value 
as the final output value. Otherwise, the final output val- 
ue will be the scythe value. Since the pixel locations 
stored in the defect memory 20 comprise only a small 
proportion of the total number of pixels in the sensor ar- 
ray, scythe filtering will be applied to the majority of the 
data stream with ring median filtering being applied to 
the remainder. 

[0039] In Fig. 4, the defect memory 20 of Fig. 2 is rep- 
resented by memory block 54 and memory manage- 
ment block 56. 

[0040] The pixel locations stored in the defect mem- 
ory 20 are those which exhibit the most extreme differ- 
ences from their neighbours. In the embodiment of Fig. 
4, pixel locations are selected for inclusion in the defect 
memory on the basis of the magnitude of the difference 
between the value of p(c) and the scythe value output 
from block 50. The difference between the two values 
is determined at 58 and the absolute magnitude of this 
difference at 60. The decision as to whether a particular 
pixel location should be stored can be based on a wide 
variety of criteria : depending in part on the size of the 
defect memory and on the memory management strat- 
egy employed. I n the present example, a simple scheme 
is employed whereby the single worst defect (i.e. the 
greatest difference between the value of p(c) and the 
scythe value) in each video frame is stored in the defect 
memory. For each Irame, the worst defect to date is 
stored in buffer memory 62. At the end of the frame, the 
value stored at 62 is passed to the memory block 54, 
together with its corresponding location in the sensor ar- 
ray. The data stored in the memory 54 is essentially a 
sorted list of pixel locations and associated defect mag- 
nitudes. Additional information could be stored if neces- 
sary. 

[0041] it will be understood that the beginnings and 
endings of video frames and the locations of pixels cor- 
responding to pixel values in the data stream can be de- 
rived by the use of clocks, counters and information in- 
cluded in the data stream, in a manner which will be fa- 
miliar to those skilled in the art. Systems for performing 
these functions will not be described herein and are ex- 
cluded from the drawings lor the sake of clarity. 
[0042] The memory management unit 56 controls the 
output multiplexer 52 so as to select the ring median val- 
ue as the final output when the current pixel corresponds 
to a location stored in the memory block 54. Otherwise, 
the scythe value is selected. 

[0043] As noted above, a strategy is required for man- 
aging the contents of the memory block 54. This is ac- 
complished in the present example by means of a first- 
order auto-regression function (also known as "leaky in- 
tegration"). That is : the magnitudes of the defects stored 
in the memory are continually updated by means of the 
auto-regression formula. Once the memory 54 is full, the 
locations with lowest defect magnitudes can be re- 



placed by newly detected defects of greater magnitude. 
The magnitudes of persistent defects will be refreshed 
by normal operation of the filtering system, whilst the 
stored magnitudes of transient defects will gradually at- 
tenuate until they are replaced. 
[0044] In this example, the magnitudes of stored de- 
fects are updated by determining the difference be- 
tween the current pixel value p(c) and the ring median 
value at 64, and the absolute magnitude of this differ- 
ence at 66. The updated value is calculated using the 
auto-regression formula at 68, from the current stored 
value for the relevant pixel location and magnitude of 
the difference between p(c) and the ring median value, 
and the stored value is updated accordingly. The loca- 
tion of the current, lowest stored value is stored in mem- 
ory buffer 70 so that this value (MIN) can be replaced 
by a new defect location and value (MAX, 62) once the 
memory 54 is full. 

[0045] It can be seen that Fig. 2 represents a gener- 
alised version of the preferred embodiment, employing 
a stored list of defect locations to apply two stage filter- 
ing to an incoming data stream, with the first stage fil- 
tering also serving to determine which locations are 
stored and the second stage filtering being switched on 
and off on the basis of the stored list. As seen in Fig. 4, 
this functionality is implemented by applying both filter- 
ing functions in parallel and selecting which filter output 
to use on the basis of the stored list, with the first stage 
filter output also being employed in the selection of lo- 
cations for storage and the second stage filter output 
also being employed in the management of the stored 
list. 

[0046] Other variations of the described embodi- 
ments can be envisaged, using different filtering func- 
tions, different data sampling schemes and different 
memory management strategies. Such variations and 
other modifications and improvements may be incorpo- 
rated without departing from the scope of the invention 
as defined in the Claims appended hereto. 



40 

Claims 

1. A method of processing a video data stream com- 
45 prising a series of pixel values corresponding to pix- 
el sites in an electronic imaging device so as to cor- 
rect defective pixel values, comprising filtering the 
video data stream in real time so as to correct or 
modify defective pixel values. 

so 

2. A method as claimed in Claim 1 , wherein the filter- 
ing of each pixel value is based on the values of a 
plurality of neighbouring pixel values. 

ss 3. A method as claimed in Claim 2, wherein the filter- 
ing of each pixel value uses the value of the current 
pixel as part of a dataset including the values of said 
neighbouring pixels in determining whether and/or 
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how to correct or modify the current pixel value. 

4. A method as claimed in any preceding Claim, fur- 
ther including the step of identifying those pixel val- 
ues which are most severely defective, storing the 
locations of said most severely defective pixels in a 
defect store, applying a first filtering algorithm to 
those pixels whose locations are not stored and ap- 
plying a second filtering algorithm to those pixels 
whose locations have been stored. 

5. A method as claimed in Claim 4, wherein the filter- 
ing of each pixel value is based on the values of a 
plurality of neighbouring pixel values and said first 
filtering algorithm uses the value of the current pixel 
as part o1 a dataset including the values of said 
neighbouring pixels. 

6. A method as claimed in Claim 5, wherein said first 
filtering algorithm comprises sorting the values of 
the current pixel and.of said neighbouring pixels into 
rank order and modifying the current pixel value on 
the basis of its place in said rank order. 

7. A method as claimed in Claim 6, wherein the value 
of the current pixel is modified if its rank is greater 
than or less than predetermined maximum and min- 
imum rank values. 

8. A method as claimed in Claim 7, wherein: 

the current pixel value is replaced by the value 
of the pixel having said predetermined maxi- 
mum rank value, if the current pixel value has 
a rank greater than said predetermined maxi- 
mum rank value; 

the current pixel value is replaced by the value 
of the pixel having said predetermined mini- 
mum rank value, if the current pixel value has 
a rank less than said predetermined minimum 
rank value; and 

the current pixel value is left unchanged if the 
current pixel value has a rank less than said 
predetermined maximum rank value and great- 
er than said predetermined minimum rank val- 
ue. 

9. A method as claimed in Claim 8, wherein said pre- 
determined maximum rank value is the highest 
ranking of said neighbouring pixels and said prede- 
termined minimum rank value is the lowest ranking 
of said neighbouring pixels. 

10. A method as claimed in any one of Claims 4 to 9 S 
wherein pixel locations to be stored in said defect 
store are selected on the basis of the output of said 
first filtering algorithm. 



11. A method as claimed in Claim 10, wherein the de- 
cision to store a pixel location is based on the mag- 
nitude of the difference between the current pixel 
value and the pixel value output by said first filtering 

s algorithm. 

12. A method as claimed in Claim 11 , wherein, for each 
frame of video data, the location of at least that pixel 
value having the greatest difference in magnitude 

to from the output of the first filtering algorithm is 
stored in said defect store. 

13. A method as claimed in any one of Claims 4 to 1 2, 
wherein the filtering of each pixel value is based on 

1$ the values of a plurality of neighbouring pixel values 
and said second filtering algorithm excludes the val- 
ue of the current pixel from a dataset including the 
values of said neighbouring pixels. . 

20 14. A method as claimed in Claim 1 3, wherein said sec- 
ond filtering algorithm replaces the value of the cur- 
rent pixel with the median value of said neighbour- 
ing pixels. 

25 15. A method as claimed in any one of Claims 4 to 14, 
wherein the information stored in said defect store 
includes the location of each pixel selected for stor- 
age and information indicating the severity of the 
defect. 

30 

16. A method as claimed in any one of Claims 4 to 15, 
wherein the contents of the defect store are updated 
in accordance with a predetermined memory man- 
agement algorithm. 

35 

17. A method as claimed in Claim 16, wherein said de- 
fect store includes the location of each pixel select- 
ed for storage and information indicating the sever- 
ity of the defect, and wherein said information re- 

40 garding the severity of the defect is updated on the 
basis of an auto-regression function applied to the 
current value of each stored pixel value, the current 
output from the second filtering algorithm and the 
current stored value. 

45 . ' 

18. A method as claimed in any one of Claims 4 to 1 7, 
wherein said first and second filtering algorithms 
are applied to the video data stream in parallel and 
the final output pixel value is selected from the out- 

50 puts of the first and second filtering algorithm de- 
pending on whether the corresponding pixel loca- 
tion is present in the defect store. 

19. Apparatus for processing a video data stream com- 
55 prising electronic filter means adapted to implement 

the method as defined in any one of Claims 1 to 19. 

20. Apparatus as claimed in Claim 19, comprising 
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means for sampling a video data stream in order to 
obtain a data set comprising a current pixel value 
and a plurality of neighbouring pixel values. 

21. Apparatus as claimed in Claim 20, further including 
means for sorting said neighbouring pixel values in- 
to rank order. 

22. Apparatus as claimed in Claim 21 , further including 
means for comparing the current pixel value with 
neighbouring pixel values of selected ranks and for 
generating a first filter output on the basis of said 
comparison. 

23. Apparatus as claimed in Claim 22, further including 
means for determining the median value of said 
neighbouring pixels and generating a second filter 
output equal to said median value. 

24. Apparatus as claimed in Claim 23, further including 
a defect store for storing pixel locations selected on 
the basis of said first filter output. 

25. Apparatus as claimed in Claim 23, further including 
output means for generating a final output pixel vaK 
ue selected from said first and second filter outputs 
on the basis of the contents of said defect store. 

26. An electronic imaging system including an image 
sensor array having an output connected to appa- 
ratus as claimed in any one of Claims 19 to 25. 

27. A method of filtering a video data stream comprising 
a series of pixel values corresponding to pixel sites 
in an electronic imaging device, wherein the filtering 
of each pixel value is based on the values of a plu- 
rality of neighbouring pixel values using the value 
of the current pixel as part of a dataset including the 
values of said neighbouring pixels, and wherein 
said filtering comprises sorting the values of the cur- 
rent pixel and of said neighbouring pixels into rank 
order and modifying the current pixel value on the 
basis of its place in said rank order. 

28. A method as claimed in Claim 27, wherein the value 
of the current pixel is modified if its rank is greater 
than or less than predetermined maximum and min- 
imum rank values. 

29. A method as claimed in Claim 28, wherein: 



mum rank value, it the current pixel value has 
a rank less than said predetermined minimum 
rank value; and 

the current pixel value is left unchanged if the 
5 current pixel value has a rank less than said 

predetermined maximum rank value and great- 
er than said predetermined minimum rank val- 
ue. 

10 30. A method as claimed in Claim 29, wherein said pre- 
determined maximum rank value is the highest 
ranking of said neighbouring pixels and said prede- 
termined minimum rank value is the lowest ranking 
of said neighbouring pixels. 

15 

31 . Apparatus for processing a video data stream com- 
prising electronic filter means adapted to implement 
the method as defined in any one of Claims 27 to 30. 

20 32. An electronic imaging system including an image 
sensor array having an output connected to appa- 
ratus as claimed in Claim 31 . 



25 



30 



35 



40 



45 



SO 



the current pixel value is replaced by the value 
of the pixel having said predetermined maxi- 
mum rank value, if the current pixel value has 
a rank greater than said predetermined maxi- 
mum rank value; 

the current pixel value is replaced by the value 
of the pixel having said predetermined mini- 
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